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1.  SCOPE. 

This  TOP  deseribes  a  systematic  approach  to  Line-of-Sight/Non  Line-of-Sight  (LOS/NLOS) 
testing  of  Unmanned  Ground  Vehicles  (UGVs).  The  objective  is  to  ensure  that  testing  of  each 
UGV  includes  measures  to  enhance  system  safety,  and  that  hazards  which  could  reduce  system 
safety  are  eliminated  or  controlled  to  an  acceptable  level  of  risk. 

1.1  Purpose. 

The  purpose  of  this  document  is  to  provide  a  systematic  approach  to  LOS/NLOS  testing  of 
UGVs  in  order  to  ensure  that  the  overall  safety  of  the  system  is  adequate  and  that  the 
performance  of  the  UGV  meets  with  expectations.  This  document  provides  uniform  procedures 
that  may  be  applied  to  the  testing  of  UGVs.  Test  results  should  be  thorough  enough  to  identify 
hazards  of  a  system  and  to  verify  performance  of  the  system  within  system  requirements. 

1.2  Applicability. 

This  document  applies  to  remote-controlled  and  teleoperated  ground  vehicles. 

1.3  Activities  Addressed. 

The  information  contained  herein  applies  to  LOS/NLOS  testing  of  robotic  systems.  Tailoring  of 
test  procedures  to  the  specific  needs  of  the  system  and  its  intended  use  are  encouraged. 

1.4  Limitations. 

This  document  applies  to  LOS/NLOS  testing  of  UGVs  only.  Methodologies  for  testing  of 
Unmanned  Aerial  Vehicles  (UAVs)  or  Unmanned  Submersible  Vehicles  (US Vs)  are  not  covered 
in  this  document.  This  document  does  not  apply  to  automated  subsystems  that  do  not  directly 
affect  people  or  objects  outside  of  the  vehicle  such  as  automated  internal  weapon  loading 
systems  or  automated  tracking  subsystems,  nor  to  automated  driver-assist  functions  that  require 
full-time  on-board  driver  attention  to  perform  a  task  such  as  cruise  control,  anti-lock  brake 
systems,  or  self-leveling  systems. 

2.  L AGILITIES  AND  INSTRUMENTATION. 

2.1  Test  Lacilities  and  Conditions. 

Testing  should  be  conducted  with  exclusive  use  of  test  courses  to  mitigate  potential  safety 
hazards  to  personnel,  buildings,  and  test  items. 


2 


TOP  2-2-543 
24  November  2008 


2.2  Instrumentation. 

a.  Instrumentation  requirements  for  LOS/NLOS  testing  of  UGVs  will  rely  extensively  on 
sensors  to  provide  information  regarding  the  environment  in  whieh  the  UGV  is  operating. 
Instrumentation  should  be  employed  to  measure  the  following: 

(1)  Vehiele  and  OCU  position  on  test  course  (±1  m). 

(2)  Communications  latency  between  OCU  and  vehicle  (±1  microsecond) 

(3)  RF  environment 

(4)  Communications  signal  strength 

b.  A  Global  Position  System  (GPS)  device  with  the  highest  possible  resolution  will  be 
used  to  mark  the  positions  of  the  OCU  and  vehicle.  The  GPS  will  be  used  to  determine  the 
distance  at  which  LOS/NLOS  communications  between  the  OCU  and  vehicle  are  lost. 

c.  In  general,  instrumentation  will  continue  to  be  used  in  traditional  physical  test  roles  to 
gather  pertinent  component  level  data  using  accelerometers,  thermocouples,  etc. 

d.  Safety  measures  for  UGV  testing  outlined  in  TOP  2-2-540^*  should  be  adhered  to  for 
LOS/NLOS  testing. 

e.  Prior  to  testing,  a  digital  spectrum  analyzer  will  be  used  to  examine  the  spectral 
composition  of  radio  frequency  (RF)  in  the  testing  area.  The  data  collected  will  be  evaluated  to 
ensure  that  RF  signals  in  the  testing  area  will  not  cause  interference  with  the  test  item. 

3.  REQUIRED  TEST  CONDITIONS. 

a.  Prior  to  the  start  of  testing,  all  test  personnel  will  receive  operator  familiarization  and 
training  on  the  test  item. 

b.  All  test  personnel  will  be  briefed  on  any  potential  hazards  associated  with  the  specific 
test  item.  All  guidelines  for  standoff  distances  and  other  safety  related  issues  will  be  followed. 

c.  Test  personnel  will  maintain  contact  with  the  test  course  supervisors  during  testing. 

All  Standing  Operating  Procedures  (SOPs)  regarding  test  course  use  will  be  followed. 

d.  Prior  to  testing,  entry  points  to  the  test  course  will  be  blocked  with  barricades  and 
signs  reflecting  robotic  testing  in  progress.  Range  lights  will  be  turned  on  where  available. 

e.  Video  coverage  of  the  test  event  will  be  provided  as  necessary. 


Superscript  numbers  correspond  to  those  in  Appendix  B,  References. 
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f.  Photographs  of  the  test  set  up  will  be  obtained. 

4.  TEST  PROCEDURES. 

a.  EOS. 

(1)  Perform  this  test  along  a  flat,  paved  straightaway  in  an  open  area  free  from 
overhanging  trees  or  other  obstaeles  that  might  impede  the  RE  connection  between  the 
Operator’s  Control  Unit  (OCU)  and  vehicle. 

(2)  Perform  an  operational  check  prior  to  testing  to  determine  baseline  status. 
Evaluate  system  performance  throughout  testing,  and  perform  operational  checks  at  least  every 
61m  (200  ft).  Perform  an  operational  check  at  the  completion  of  testing  to  determine  any 
degradation  of  performance. 

(3)  The  UGV  will  be  situated  beside  or  directly  in  front  of  the  OCU. 

(4)  The  UGV  will  be  driven  away  from  the  OCU  at  a  speed  no  greater  than  16  km/hr 
(10  mph)  to  the  maximum  required  EOS  operating  distance  or  until  a  degraded  mode  is  indicated 
by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(5)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(6)  Record  the  distance  between  the  OCU  and  the  vehicle  at  the  time  of  the  failure. 

(7)  Repeat  the  above  procedure  at  least  three  times  for  comparison  purposes  and  to 
establish  consistency  of  system  performance. 

b.  NCOS,  natural  terrain.  During  all  NCOS  testing,  a  secondary  safety  driver  should  be 
used  to  maintain  EOS  with  the  UGV  at  all  times. 

(1)  In  order  to  establish  the  NLOS  natural  terrain  environment,  conduct  testing  in  an 
area  containing  a  natural  obstruction.  The  test  may  be  conducted  along  the  curve  of  a  course, 
with  thick  foliage  present  along  the  inside  of  the  curve,  as  shown  in  Eigure  1 . 

(2)  Perform  an  operational  check  prior  to  testing  to  determine  baseline  status. 
Evaluate  system  performance  throughout  testing,  and  perform  operational  checks  at  least  every 
16  m  (200  feet).  Perform  an  operational  check  at  the  completion  of  testing  to  determine  any 
degradation  of  performance. 
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Natural  terrain  NLOS 
obstacle 


OCU 


VEHICLE 


Figure  1.  Natural  obstaele  for  NLOS  operations 

(3)  Doeument  the  density  and  composition  of  the  natural  obstacle  with  photographs. 

(4)  The  UGV  will  be  situated  beside  or  directly  in  front  of  the  OCU  at  the  beginning 
of  the  curve  in  the  test  course. 

(5)  The  UGV  will  be  driven  away  from  the  OCU  (as  shown  above  in  Figure  1)  at  a 
speed  no  greater  than  16  km/hr  (10  mph)  to  the  maximum  required  NLOS  natural  terrain 
operating  distance  or  until  a  degraded  mode  is  indicated  by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(6)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(7)  Record  the  distance  between  the  OCU  and  the  vehicle  at  the  time  of  the  failure. 

(8)  Repeat  the  above  procedure  at  least  three  times  for  comparison  purposes  and  to 
establish  consistency  of  system  performance. 
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e.  NLOS,  urban  environment.  During  all  NLOS  testing,  a  secondary  safety  driver  should 
be  used  to  maintain  LOS  with  the  UGV  at  all  times. 

(1)  The  NLOS  urban  environment  should  consist  of  several  buildings  or  simulated 
buildings  situated  close  together  with  network  of  paths  or  roads  between  them.  For  the  purpose 
of  test  documentation,  the  dimensions  of  each  building,  building  materials  used  in  each  building, 
distances  between  the  buildings,  the  length  of  each  path  or  road,  and  GPS  coordinates  throughout 
the  test  site  should  be  recorded. 

(2)  Perform  an  operational  check  prior  to  testing  to  determine  baseline  status. 

Evaluate  system  performance  throughout  testing,  and  perform  operational  checks  at  least  every 
61m  (200  ft).  Perform  an  operational  check  at  the  completion  of  testing  to  determine  any 
degradation  of  performance. 

(3)  The  UGV  will  be  situated  beside  or  directly  in  front  of  the  OCU.  Test  iterations 
will  be  conducted  both  with  the  OCU  positioned  outside  the  urban  environment  and  with  the 
OCU  positioned  inside  the  urban  environment. 

(4)  The  UGV  will  be  driven  away  from  the  OCU  and  around  buildings  at  a  speed  no 
greater  than  10  mph  to  the  maximum  required  NLOS  urban  environment  operating  distance  or 
until  a  degraded  mode  is  indicated  by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(5)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(6)  Using  a  GPS,  record  the  linear  distance  between  the  OCU  and  the  vehicle  at  the 
time  of  the  failure.  Record  the  number  of  buildings  between  the  OCU  and  vehicle  at  the  time  of 
the  failure,  as  well  as  the  dimensions  of  each  building  and  the  building  materials  that  comprise 
each  building  between  the  OCU  and  vehicle. 

(7)  Repeat  each  above  procedure  at  least  three  times  for  comparison  purposes  and  to 
establish  consistency  of  system  performance. 

d.  NLOS,  convoy  operations  (OCU  stationary,  vehicle  stationary).  During  all  NLOS 
testing,  a  secondary  safety  driver  should  be  used  to  maintain  LOS  with  the  UGV  at  all  times. 

(1)  Convoy  operations  should  be  conducted  on  a  flat,  paved  test  course,  as  well  as  on 
secondary  (dirt)  roads  and  cross-country  trails.  For  this  test,  operations  will  be  executed  with  the 
OCU,  UGV,  and  all  convoy  vehicles  stationary  in  order  to  establish  a  baseline  against  which 
moving  convoy  test  results  can  be  measured. 
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(2)  Perform  an  operational  check  prior  to  testing  to  determine  baseline  status. 
Evaluate  system  performance  and  perform  operational  checks  throughout  testing.  Perform  an 
operational  check  at  the  completion  of  testing  to  determine  any  degradation  of  performance. 

(3)  Identify  vehicles  that  would  typically  convoy  with  the  test  item  in  an  operational 
scenario.  For  the  purpose  of  this  example,  the  convoy  will  consist  of  commonly  used  small  and 
large  vehicles;  the  High  Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV)  may  be  used  to 
represent  a  commonly  used  smaller  vehicle,  and  the  Family  of  Medium  Tactical  Vehicles 
(FMTVs)  may  be  used  to  represent  a  commonly  used  larger  vehicle.  Instrument  all  vehicles  in 
the  convoy  with  GPS. 

(4)  Dismountable  OCU  in  open  air,  stationary  operations.  The  OCU  will  be 
stationary  and  in  the  open  air  (not  on  board  a  vehicle);  this  test  will  only  be  performed  for 
systems  wherein  the  OCU  is  not  integrated  into  a  command  vehicle,  or  is  designed  to  be 
dismountable  from  its  command  vehicle.  Place  one  intervening  vehicle  between  the  OCU  and 
the  vehicle,  and  conduct  operational  checks.  Increase  the  number  of  intervening  vehicles 
between  the  OCU  and  UGV  until  the  system  requirement  is  met  or  until  a  degraded  mode  is 
indicated  by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(5)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(6)  OCU  on  board  a  command  vehicle,  stationary  operations.  The  OCU  will  be 
stationary  and  on  board  a  command  vehicle.  Place  one  intervening  vehicle  between  the  OCU 
and  the  UGV,  and  conduct  operational  checks.  Increase  the  number  of  intervening  vehicles 
between  the  OCU  and  UGV  until  the  system  requirement  is  met  or  until  a  degraded  mode  is 
indicated  by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(7)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(8)  Using  a  GPS,  record  the  linear  distance  between  the  OCU  and  the  vehicle  at  the 
time  of  the  failure.  Record  the  number  and  type  of  vehicles  between  the  OCU  and  vehicle  at  the 
time  of  the  failure,  as  well  as  the  distances  between  each  vehicle  in  the  convoy. 
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(9)  Repeat  eaeh  above  proeedure  at  least  three  times  for  eomparison  purposes  and  to 
establish  eonsistency  of  system  performance. 

e.  NLOS,  convoy  operations  (moving  operations).  During  all  NLOS  testing,  a  secondary 
safety  driver  should  be  used  to  maintain  LOS  with  the  UGV  at  all  times. 

(1)  Convoy  moving  operations  should  be  conducted  on  a  flat,  paved  test  course,  as 
well  as  on  secondary  (dirt)  roads  and  cross-country  trails.  For  this  test,  operations  will  be 
executed  with  the  OCU,  UGV,  and  all  convoy  vehicles  moving. 

(2)  The  OCU  will  be  on  board  a  command  vehicle. 

(3)  Place  one  intervening  vehicle  between  the  OCU  and  the  vehicle. 

(4)  The  OCU  operator  will  accelerate  the  UGV.  The  command  vehicle  and  the 
intervening  vehicle  will  accelerate  with  the  UGV.  Initially,  a  distance  of  approximately  100  m 
(328  ft)  will  be  kept  between  the  UGV  and  the  rest  of  the  convoy;  this  distance  will  be  expanded 
to  at  least  300  m  (984  ft)  or  until  the  control  link  is  lost.  A  distance  of  approximately  50  m  (164 
ft)  will  be  kept  between  each  of  the  remainder  of  the  convoy  vehicles. 

(5)  An  additional  intervening  vehicle  will  be  added  to  the  convoy  after  each 
successful  convoy  operation.  This  maneuver  will  be  performed  until  the  system  requirement  is 
met  or  until  a  degraded  mode  is  indicated  by  any  of  the  following: 

(a)  The  OCU  video  feed  reverts  to  the  blue  screen  for  more  than  3  seconds. 

(b)  A  data  link  failure  occurs. 

(c)  Any  other  system  fault  (such  as  loss  of  steering  or  sensor  control)  occurs. 

(6)  Record  a  description  of  any  loss  or  degradation  of  function,  to  include  a 
categorization  of  the  severity  of  the  fault. 

(7)  Using  a  GPS,  record  the  linear  distance  between  the  OCU  and  the  vehicle  at  the 
time  of  the  failure.  Record  the  number  of  vehicles  and  type  and  position  of  each  vehicle  in  the 
convoy  at  the  time  of  the  failure,  as  well  as  the  distances  between  each  vehicle  in  the  convoy. 

(8)  Repeat  the  above  procedure  at  least  three  times  for  comparison  purposes  and  to 
establish  consistency  of  system  performance. 

5.  DATA  REQUIRED. 

a.  Documentation  will  be  provided  detailing  all  aspects  of  testing  including  the 
following: 


(1)  Vehicle  serial  number. 
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(2)  Photographs  of  test  site  and  setup. 

(3)  Video  and  data  link  operating  frequeney. 

(4)  Distanee  between  OCU  and  UGV  when  video  or  data  link  was  lost. 

(5)  Any  failure  or  limitation  of  the  operational  checklist  during  testing. 

(6)  Detailed  description  of  obstacles  to  be  used  during  NLOS  testing,  including 
photographs,  GPS  coordinates,  test  course  dimensions,  and  the  following: 

(a)  NLOS  (Natural  Terrain,  Wooded).  The  natural  terrain  should  be  represented  in 
photographs  and  otherwise  measured,  where  possible. 

(b)  NLOS  (Natural  Terrain,  Hills).  The  geodetically  surveyed  elevation  of  the  hill  or 

hills. 

(c)  NLOS  (Urban  Terrain).  The  dimensions,  layout,  and  construction  materials  that 
comprise  the  buildings. 

(d)  NLOS  (Convoy  Operations).  Provide  a  list  of  vehicles  used,  distances  between 
vehicles,  speeds  of  travel  for  all  vehicles,  and  communication  equipment  that  may  be  installed 
and  used  on  the  obstacle  vehicles  during  operations. 

6.  PRESENTATION  OF  DATA. 

a.  Describe  the  inspection,  specific  test  procedures,  and  results  for  each  item  using 
narration,  tables,  photographs,  charts,  and  graphs  as  appropriate  or  as  outlined  in  procedures  and 
data  required. 

b.  Reduce,  summarize,  and  analyze  data  from  each  subtest  appropriate  to  the  subtest  data 
topic  and  failure  definitions  derived  specifically  for  the  item  and  the  subtest  category. 
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APPENDIX  A.  ABBREVIATIONS. 


DOD 

Department  of  Defense 

EOV 

Eield  of  View 

IR 

Infrared 

JAUS 

Joint  Arehiteeture  of  Unmanned  Systems 

JGRE 

Joint  Ground  Robotics  Enterprise 

OCU 

Operator  Control  Unit 

RE 

Radio  Erequency 

SUT 

System  under  test 

TIRS 

Test  Incident  Reports 

TOP 

Test  Operations  Procedure 

UGV 

Unmanned  Ground  Vehicle 
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1.  TOP  2-2-540,  Testing  of  Unmanned  Ground  Vehicle  (UGV)  Systems,  30  June  2008. 

For  information  only  (related  publications). 

a.  TOP  2-2-542,  Safe  Operation  of  Weaponized  Unmanned  Ground  Vehicle  (UGV)  Systems, 
8  July  2008. 

b.  TOP  3-2-812,  Field  of  Vision  -  Vehicles,  23  February  1966. 

c.  Unmanned  Systems  Safety  Guide  for  DOD  Acquisition,  27  June  2007. 
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Forward  comments,  recommended  changes,  or  any  pertinent  data  which  may  be  of  use  in 
improving  this  publication  to  the  following  address:  Test  Business  Management  Division 
(TEDT-TMB),  US  Army  Developmental  Test  Command,  314  Longs  Comer  Road  Aberdeen 
Proving  Ground,  MD  21005-5055.  Technical  information  may  be  obtained  from  the  preparing 
activity:  US  Army  Aberdeen  Test  Center  (TEDT-AT-AD-F),  400  Colleran  Road,  Aberdeen 
Proving  Ground,  MD  21005-5059.  Additional  copies  can  be  requested  through  the  following 
website:  http://itops.dtc.armv.mil/RequestForDocuments.aspx,  or  through  the  Defense 
Technical  Information  Center,  8725  John  J.  Kingman  Rd.,  STE  0944,  Fort  Belvoir,  VA 
22060-6218.  This  document  is  identified  by  the  accession  number  (AD  No.)  printed  on  the 
first  page. 


